CNN Project Part 3¶
• DOMAIN: Automobile¶
• CONTEXT: A brand research company wants to understand which cars or car manufacturers are popular in a certain area of the city or locality. Company has a team which takes pictures of the cars randomly through the day. Using this the company wants to set up an automation which can classify the make of the car once the picture has been given as an input.¶
• TASK: Help to build the image dataset to be used by the AI team to build an image classifier data. Import and display the images in python against their labels. Comment on the challenges faced during this task.¶
(1) Import Python libraries¶
(1.1) Enable GPU for this notebook¶
In [1]:
import tensorflow as tf
print("You are using TensorFlow version", tf.__version__)
if len(tf.config.list_physical_devices('GPU')) > 0:
print("You have a GPU enabled.")
else:
print("Enable a GPU before running this notebook.")
from tensorflow.keras.models import Sequential # initial NN
from tensorflow.keras.layers import Dense, Dropout, InputLayer, BatchNormalization # construct each layer
from tensorflow.keras.layers import Conv2D # swipe across the image by 1
from tensorflow.keras.layers import MaxPool2D, GlobalMaxPool2D # swipe across by pool size
from tensorflow.keras.layers import Flatten, GlobalAveragePooling2D
from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.vgg16 import preprocess_input,decode_predictions
import numpy as np
import pandas as pd
import os
import cv2
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
from tqdm.notebook import trange, tqdm
import seaborn as sns
from google.colab.patches import cv2_imshow
from IPython.display import Image, display, Markdown, clear_output
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import LabelEncoder
from sklearn import svm, metrics
from sklearn.decomposition import PCA
from skimage import io
import math
(1.2) Mount google drive that contains image dataset¶
In [2]:
from google.colab import drive
drive.mount('/content/drive')
(2) Load Image dataset¶
(2.1) Set the globval variables for project and image dta paths¶
In [3]:
image_path = '/content/drive/MyDrive/Colab Notebooks/DataFile/Cars Images/'
project_path = '/content/drive/MyDrive/Colab Notebooks/'
(2.2) Read the dataframe conatining image file names and model details¶
In [4]:
cars_df = pd.read_csv('/content/drive/MyDrive/Colab Notebooks/cars_data_set.csv')
cars_df
Out[4]:
In [5]:
cars_df.info()
Observations:¶
- There are 15 images of cars along with the make, model , segment they belong to and their manufactured year.
(2.2.1) Convert the manuafacture_year column from int to str¶
In [6]:
cars_df['manufacture_year'] = cars_df['manufacture_year'].astype('str')
In [7]:
cars_df['Description'] = cars_df['Make']+' '+cars_df['model'] + ' ' + cars_df['segment'] + ' ' + cars_df['color'] + ' ' + cars_df['manufacture_year']
cars_df
Out[7]:
(3) Visualize images¶
(3.1) Set the resizing image size¶
In [8]:
img_sz = 224
(3.2) Display all images with their labels from the dataframe.¶
In [9]:
cols = 4
rows = int(math.ceil(len(cars_df)/cols))
fig, ax = plt.subplots(rows,cols,figsize=(20,20))
X = []
y = []
for index in range(0,len(cars_df)):
img = cv2.imread(image_path+cars_df['file_name'][index])
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img,(img_sz,img_sz),interpolation = cv2.INTER_NEAREST)
X.append(img)
y.append(cars_df['segment'][index])
if index < cols:
col = index
row = int(index/4)
else:
col = index % 4
row = int(index/4)
ax[row][col].imshow(img)
ax[row][col].set_title(f"{cars_df['Description'][index]}",fontsize=15)
ax[row][col].tick_params(top=False, bottom=False, left=False, right=False,labelleft=False, labelbottom=False)
plt.show()
X = np.array(X)
y = np.array(y)
In [10]:
display(Markdown(f"#### Shape of image set is {X.shape} and {y.shape}"))
(4.1) Use the VGG16 pre-trained model to predict the type of cars¶
In [11]:
from keras.applications.vgg16 import VGG16
model = VGG16(weights='imagenet')
print(model.summary())
Observations:¶
- As we could see VGG16 has a total of 138M trainable parameters.
- All these parameters are pre-trained using
imagenet. - Also, the output layer has
1000distint classes to choose from
(4.2) Predict the type of car from the pictures and add it to the original dataframe¶
In [15]:
vgg_predict = []
for img in X:
img = np.expand_dims(img,axis=0)
img = preprocess_input(img)
features = model.predict(img)
p = decode_predictions(features)
vgg_predict.append(p[0][0][1])
cars_df['VGG16-Label'] = vgg_predict
cars_df
Out[15]:
(4.3) Visualise the images with the true label and prediction¶
In [19]:
cols = 4
rows = int(math.ceil(len(cars_df)/cols))
fig, ax = plt.subplots(rows,cols,figsize=(20,20))
for index in range(0,len(cars_df)):
img = cv2.imread(image_path+cars_df['file_name'][index])
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img,(img_sz,img_sz),interpolation = cv2.INTER_NEAREST)
if index < cols:
col = index
row = int(index/4)
else:
col = index % 4
row = int(index/4)
ax[row][col].imshow(img)
ax[row][col].set_title(f"y : {cars_df['segment'][index]} | y-hat : {cars_df['VGG16-Label'][index]}",fontsize=15)
ax[row][col].tick_params(top=False, bottom=False, left=False, right=False,labelleft=False, labelbottom=False)
plt.show()
Observations:¶
- We could see that not all labels have been predicted correctly , especially the car
Geo Metrowhich actually is aconvertiblebut predicted as amobile_home. This could be due to the reason that there is a home like structrure in the background and which dominates tha image of the car itself.
(5) Conclusion:¶
(5.1) Challenges faced:¶
- Cars are in general identified by their logo in the front or rear and then by their physical appearence except a few luxuary lineup. If the images do not capture this details then it would be difficult to label them.
- The background features most often also play a dominant role i.e if a car of interest has many other cars of different make or model in the background then the images can be potentially misidentified.
- There are plenty of resources in the internet but to search them by picture is challenging without a proper label or logo on the car.
(5.2) Solutions:¶
- Used Google Lens capability in our android phone to search for cars in the image dataset and get their details and this cam eto our rescue
- To avoid misclassification due to background images and noices, we had to crop the image of car.
In [7]:
!pwd
In [9]:
%%shell
jupyter nbconvert --to html --template full /content/drive/MyDrive/Colab Notebooks/CNN_project_3.ipynb
In [ ]:
In [ ]: